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Abstract 
Children from low-income families are at greater risk for poor social-emotional development and 
physical health and may be in need of intervention. This study examined the extent to which the 
Positive Action (PA) preschool lessons improved low-income children’s social-emotional 
competence and health behaviors. Mixed findings emerged with regard to whether the lessons 
facilitated growth in child outcomes. Results showed positive effects of PA on children’s directly 
assessed social problem solving skills and their parent-rated social-emotional competence and 
health behaviors. Unexpectedly, program effects on teacher ratings of social-emotional 
competence were in the opposite direction. These findings provide preliminary evidence that PA 
may be effective in promoting positive social-emotional and health behavior outcomes for 
children, particularly when outcomes are assessed directly and by parents. 
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The effects of Positive Action on preschoolers’ social-emotional competence and health 
behaviors 

Evidence suggests that children from low-income families begin kindergarten at a 
disadvantage in their school readiness, including academic skills, social-emotional competence, 
and physical health (Bradley & Corwyn, 2002; Qi & Kaiser, 2003; Woodfield, Duncan, Al- 
Nakeeb, Nevill, & Jenkins, 2002), which may contribute to long-term social-emotional, 
educational, and health disparities (Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011). 
Research indicates that comprehensive, classroom-based interventions can facilitate positive 
growth in young children’s social-emotional readiness for school (Nix et al., 2016); however, 
less is known regarding whether these types of interventions may also have an impact on 
children’s health-related behaviors (e.g., fruit and vegetable consumption). Emerging evidence 
suggests that social-emotional development and health-related behaviors may be related in young 
children (Hughes, Power, O’Connor, & Fisher, 2015), and thus, addressing these two domains 
simultaneously through classroom-based intervention may be a fruitful endeavor. In this study, 
we explore whether the Positive Action (PA) preschool program, a classroom-based, social- 
emotional learning and health promotion program, has an impact on children’s social-emotional 
competence and health behaviors. 
Importance of Developing Social-Emotional Competence and Healthy Behaviors 

Social-emotional competence is broadly defined as the development of a child’s capacity 
to behave appropriately in social situations by regulating his/her own emotions (Denham, 2006). 
The development of social-emotional skills begins in infancy, and continues in early childhood 
when children begin developing relationships with peers and adults in multiple settings. Studies 


have shown that the development of strong social-emotional competence in preschool is 
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necessary for successful interactions with peers and adults, and such interactions may lead to 
better school adjustment and academic success (Ladd, 1999). In contrast, deficiencies in early 
social-emotional skills are linked with behavioral problems such as aggressive tendencies, 
bullying, and poor problem solving (Laursen & Pursell, 2009). 

Physical health is also an important predictor of school readiness and educational 
outcomes. Evidence suggests that children who have developed healthy behaviors (e.g., 
consumption of healthy foods) tend to have better physical health as well as stronger grades and 
more adaptive classroom behaviors (Fedewa & Ahn, 2011). In contrast, children who have not 
developed healthy behaviors are at risk for physical health problems and difficulties in school 
(Florence, Asbridge, & Veugelers, 2008). Thus, it is likely important to help children develop 
healthy behaviors when designing effective school-based interventions. Although research 
suggests that parents play a vital role in establishing preschool children’s behavioral patterns 
(Benton, 2004), literature also documents that programs implemented in preschool settings can 
have a significant impact on health behaviors (Joseph, Gorin, Mobley, & Mobley, 2015). 

An emerging body of research suggests that social-emotional competence and health may 
be related constructs. For example, evidence suggests that preschool children’s weight status is 
associated with their social skills and behavioral adjustment over time (Tandon, Thompson, 
Moran, & Lengua, 2015). This is not surprising given that behavioral patterns in both the social 
and physical health domains are developing during the early childhood period (Birch & Fisher, 
1998). Further, with samples of older children and adults, health-related behaviors, such as 
physical activity and consuming healthy and nutrient-rich foods are related to emotional 
intelligence and learning (Basch, 2011; London & Castrechini, 2011). As such, targeting these 


two constructs in classroom-based interventions may be an effective approach for promoting 
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school readiness and long-term academic success. 
Disparities in Social-Emotional Competence, Health Behaviors, and School Readiness 

In the United States, more than 1 in 5 children live in poverty (U.S. Census Bureau, 
2011). Studies suggest that children from low-income families may be at elevated risk for both 
physical health problems and poor social-emotional development, which can lead to behavior 
problems and classroom difficulties (Qi & Kaiser, 2003). These deficits in important school 
readiness skills may play a significant role in the achievement gap between poor and non-poor 
children that is already present at kindergarten entry. Although children from low-income 
families may struggle with school readiness skills and health behaviors, they also typically 
benefit the most from interventions. In a review on behavioral interventions in preschool, 
Diamond and Lee (2011) reported that, in general, children coming from lower-income families 
experience the greatest improvements in school readiness as a result of intervention participation. 
Intervention Research 

Recent intervention work indicates that social-emotional skills are malleable in preschool 
and can be taught through classroom-based lessons (Bierman, Nix, Greenberg, Blair, & 
Domitrovich, 2008). For example, the PATHS curriculum (used as part of the Head Start REDI 
curriculum) includes a series of lessons that are designed to promote preschool children’s social- 
emotional competence and reduce problem behaviors. Several studies have indicated short- and 
longer-term positive effects of PATHS on preschool children’s social competence as well as 
their emotion knowledge (Bierman et al., 2008; Nix et al., 2016). Similarly, there is evidence that 
children’s health-related behaviors can be improved through intervention (Dauenhauer, Keating, 
& Lambdin, 2016), although few studies have focused on preschool children (Wolfenden et al, 


2012). Data from some of this work suggests that health-related behaviors, such as fruit and 
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vegetable consumption, can be promoted through classroom-based interventions implemented at 
preschools (Witt & Dunn, 2012). From a health promotion perspective, theoretical frameworks, 
such as the Theory of Triadic Influence, suggest that interventions designed to improve health 
behaviors likely have greater effects when more distal and proximal factors, such as social- 
emotional components, are also included (Flay, Snyder, & Petraitis, 2009). Notably, PA has been 
effective in promoting health-related behaviors for older children (Bavarian et al., 2016). 

The Positive Action (PA) Program. Taken together, evidence suggests that intervention 
and prevention efforts that target the domains of social-emotional competence and health 
behaviors may be particularly important for children from high-risk backgrounds (e.g., low- 
income families). The PA program focuses on promoting and strengthening positive behaviors 
that foster social-emotional, intellectual, and physical health (Flay & Allred, 2010). PA is guided 
by social learning as well as social cognitive theories and prior evidence on effective strategies 
for changing children’s behavior. These theories posit that in order to see behavioral change, 
children must develop strong self-efficacy and learn to accurately interpret environmental cues 
and effectively deal with social problems (Luszczynska & Schwarzer, 2005). PA aims to build 
children’s self-efficacy and social-emotional competence by giving them the tools and 
confidence to be able to engage in positive behaviors, including health-related behaviors. Extant 
literature has demonstrated the efficacy of PA for a variety of outcomes, including short-and 
long-term academic achievement (Flay & Allred, 2010), prosocial behavioral trajectories (Lewis 
et al., 2016), emotional well-being (Lewis et al., 2013), and health behaviors (Bavarian et al., 
2016). Given these beneficial effects for older children and adolescents, and a recent focus on the 
importance of integrating social-emotional learning into preschool programs (Jones & Bouffard, 


2012), the PA creators developed a curriculum designed for preschool classrooms. 
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To date, just one evaluation has been conducted on the preschool program (Schmitt et al., 
2014). Results from this pilot study indicated that teachers found program implementation 
feasible and children were highly engaged, enjoyed the activities, and were able to understand 
the content. In addition, results suggested that the PA preschool program lessons were effective 
in improving teacher ratings of children’s social-emotional competence and behavior. However, 
there were several limitations to this preliminary evaluation, including a quasi-experimental 
design, utilization of just one teacher-reported measure of children’s skills and behaviors, and 
limited fidelity of implementation assessment. 
The Current Study 

Using a randomized controlled design, the current study examined whether: 1) children 
who participated in PA demonstrated improvements across several social-emotional competence 
and health behaviors measures, including parent reports, teacher reports, and a direct assessment, 
and 2) teachers could effectively implement PA with fidelity and engage children in PA lessons. 
It was hypothesized that relative to a control group, children who participated in PA would 
demonstrate greater improvements in their social-emotional competence and health behaviors. It 
was also expected that fidelity of implementation would be high. 

Method 

Participants 

The current sample consisted of 75 children (47% female; n = 40 in intervention 
classrooms, 1 = 35 in control classrooms) enrolled in 10 preschool classrooms across 5 sites 
located in the Midwestern region of the United States. Informed consent was obtained from the 
parents of all children participating in the study. Consent forms were sent home to children in all 


classrooms, and approximately 50% of consent forms were returned. All of the sites primarily 
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served low-income children (e.g., 3 of the 5 sites were Head Start centers; 2 of the sites were 
community-based child care centers in low-resource neighborhoods); however, average parent 
education was significantly lower at Head Start centers (M = 3.08, SD = 1.05 versus M = 5.09, 
SD = 1.86; t(72)=5.99, p<.001). Children and families were recruited through letters sent home 
in the fall of the preschool year or through parent recruitment nights. Children ranged in age 
from 30 to 62 months (M = 51.07, SD = 6.70). Sixty-four percent of the sample was Caucasian, 
6% were African American, 8% were Hispanic, and 22% were Multiracial. Twenty percent of 
parents in the sample had less than a high school diploma, 35% had a high school diploma or 
GED, 23% had some college, and 22% had a college or graduate degree. Participating families 
received a $20 gift card, and participating teachers received a $40 gift card. All teachers reported 
that they did not currently use a social-emotional learning or health curriculum in their 
classrooms. 
Procedure 

Prior to the study period, the Principal Investigator worked with PA staff to develop an 
implementation schedule of lessons that fit the study duration for implementation (15 weeks). 
Research assistants that administered direct assessments were blind to condition. Furthermore, in 
an attempt to keep parents blind to condition, the research team did not communicate assignment 
to parents, and teachers were asked to not share this information with parents as well. In the fall 
at study baseline, parents were asked to complete demographic questionnaires. 

Pre-test and post-test data collection. In the fall (prior to intervention implementation) 
and again in the spring (following intervention implementation), parents and teachers were asked 
to rate children’s social-emotional competence as well as health behaviors on the Student Rating 


Scale (Schmitt et al., 2014). Parents also completed a separate scale measuring child health 
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behaviors. Teachers were asked to rate children’s social skills and problem behaviors on the 
Social Skills Improvement Rating Scale (Gresham & Elliot, 2008). Finally, a direct assessment 
measuring children’s social problem solving skills (Aber, Brown, Jones, & Samples, 1995) was 
administered by trained research assistants and took approximately 10 minutes to complete. 

Intervention. Following pre-test data collection, teachers were randomly assigned to a 
Positive Action (PA; intervention) condition or a control condition (business-as-usual). Control 
teachers were asked to conduct their classrooms as they would normally and to not introduce 
new curricula focused on social-emotional learning or health during the study. After study 
completion, all control teachers received the intervention materials. Teachers in the treatment 
condition (n = 5) attended a 3-hour PA training workshop hosted by PA staff via webinar. 
Teachers also received classroom kits containing materials and detailed lesson plans. During the 
training, teachers were first introduced to the PA model, philosophy, and six units. Then, 
teachers learned about previous research and evaluations of PA and associated outcomes. Finally, 
the teachers worked with the materials in the classroom kit and became familiar with the lessons 
and content. Immediately following the training workshop, teachers completed an evaluation of 
the training. On a 5-point scale (1 = poor, 5 = excellent), teachers rated the trainer, workshop 
content, and overall training. Overall, teacher ratings were high across these domains (trainer 
= 4.50; workshop content M = 4.75; overall training M = 4.75). After the initial training, teachers 
in the intervention condition implemented the PA curriculum and teachers in the control 
condition delivered business-as-usual curricula. Control teachers reported implementing no 
social-emotional or health-focused curricula during the study. 

Over the course of 15 weeks, intervention teachers implemented 64, 10-15 minute daily 


scripted lessons that were integrated into the regular curriculum. All children in the intervention 
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classrooms participated in the lessons; however, pre- and post-test data were only collected on 
those with parent consent. The lessons were based on six units: Unit 1: understanding of PA and 
self-concept; Unit 2: physical health and intellectual health; Unit 3: self-management and self- 
control; Unit 4: respect, considerate, and social bonding; Unit 5: honesty with self and others; 
and Unit 6: self-improvement (see Schmitt et al., 2014 for an overview of unit themes and goals). 
Each unit builds on the prior unit(s) and becomes slightly more complex. Lessons utilized a 
variety of age-appropriate strategies and methodologies (e.g., puppets, manipulatives, games, 
music, stories; Allred, 2009). Within all lessons, there are opportunities for children to engage 
with the materials and discuss concepts to ensure active learning. For a more comprehensive 


description of the lessons, see the Positive Action website (https://www.positiveaction.net/). 


Measures 

Student Rating Scale. The Student Rating Scale (Schmitt et al., 2014), a parent and 
teacher report, consists of 33 items assessing the different social-emotional domains and health 
behaviors that the PA program addresses (i.e., understanding of PA, self-concept, health 
behaviors, intellectual health, self-management, self-control, respect, consideration, social 
bonding, honesty, and self-improvement). Accordingly, the Student Rating Scale consists of 11 
subscales, consisting of three items each. For each item, parents and teachers were asked to rate 
how much the item described the child’s behavior on a 7-point scale from not at all (1) to very 
much (7). Example items include “Follows rules, accepts limits, and cleans up his/her mess when 
asked” (self-management) and “Likes to eat healthy food and avoid unhealthy foods” (health 
behaviors). A mean score was calculated for the total scale (all 33 items). The responses for three 
negatively worded items were reversed before calculating the average, so that higher scores on 


all scales represented better behavior. This scale has demonstrated strong reliability in previous 
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work (Schmitt et al., 2014) and in the current study (parent-report a pre-test = .94; post-test = 
.97; teacher-report a pre-test = .97; post-test = .96). 

Social Skills Improvement System-Rating Scale (SSIS-RS). The SSIS-RS is a teacher 
report that assesses children’s social skills and problem behaviors (Gresham & Elliot, 2008). The 
scale consists of 76 items (46 for the social skills domain and 30 for the problem behaviors 
domain). Example items include: “interacts well with other children” (social skills domain) and 
“gets distracted easily” (problem behaviors domain). Frequencies of behaviors are scored on a 4- 
point scale (0 = never, 3 = almost always). Mean scores were created for the two scales: social 
skills and problem behaviors. The scales demonstrated strong internal consistency in the sample 
(a social skills pre-test = .97, post-test = .94; problem behaviors pre-test = .94, post-test = .96). 

Health behaviors scale. Parents completed an 11-item survey assessing the frequency of 
their child’s health behaviors. This measure was adapted from the student self-report version 
utilized in a prior study of PA (Bavarian et al., 2016). Example items include: “my child eats 
fresh fruits and vegetables” and “my child exercises hard enough to make him/her sweat and 
breathe hard.” Frequencies of behaviors were scored on a 4-point scale (1 = never, 4 = always). 
A mean score was created to represent the frequency of health behaviors, with higher scores 
indicating higher frequency of healthy behaviors. This assessment demonstrated good internal 
consistency in the current sample (a pre-test = .73, post-test = .78). 

Social problem solving. A direct assessment was administered that measured each child’s 
social problem solving skills (Aber et al., 1995). Children were presented with five vignettes and 
associated pictures describing various social scenarios. After the experimenter read each vignette 
aloud, the child was provided with five different responses to each scenario and chose which best 


described how he or she would respond. An example of one vignette is: “Pretend this is YOU 


11 


POSITIVE ACTION, SOCIAL-EMOTIONAL SKILLS, AND HEALTH BEHAVIORS 


and this is a boy or girl in your class. The other child has been on the swing for a long, long time 
and doesn’t seem to want to share the swing with you. You would really like to play on the 
swing. What would you say or do so that you could play on the swing? Would you ...” 
Response choices were as follows: A. say, “You'd better let me play?” B. ask them to share the 
swing? C. ask the teacher to make them get off the swing? D. tell the teacher to not let them play 
anymore? E. just leave? One variable was created from each vignette to assess whether 
children’s responses were aggressive or not (0 = not aggressive response; | = aggressive 
response). For example, in the vignette provided above, answers B, C, D, and E were scored as 0, 
and A was scored as 1. Inter-item consistency was fairly low for this scale (a pre-test = .36, post- 
test = .55); however, there were only five items and the frequency with which children responded 
aggressively was relatively low, both of which have an impact on Cronbach’s alpha. 

Teacher implementation survey. Each week, intervention teachers completed an 
implementation survey that included questions related to curriculum delivery and attitudes about 
the PA program. An example item related to curriculum delivery included: “How many lessons 
did you teach this week?” Items also included frequency questions, such as “How often did you 
deliver the lessons as was intended?” These items were rated on a 4-point scale (1 = rarely, 4 = 
always). Example items related to attitudes about PA included: “The more effort I put into the 
Positive Action lessons, the more effective they are” and “The time required by Positive Action is 
well worth it in improved student behavior and classroom management.” These items were rated 
on a 4-point scale (1 = strongly disagree, 4 = strongly agree). 

Observation of implementation fidelity. Independent observers visited intervention 
classrooms biweekly to assess adherence and quality of implementation as well as child 


engagement. Observers rated adherence and quality of implementation on several items on a 3- 
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point scale ranging from not at all (0) to very much (3). An example item of implementation 
adherence is: “Teacher followed lesson procedures.” An example item of implementation quality 
is: “Teacher kept a positive tone throughout the lesson.” Observed child engagement was 
assessed using similar procedures as Ling and Barnett (2013). Throughout the lesson, observers 
used a stopwatch to track time. After each one-minute interval, observers would scan the room 
and count the number of children who were engaged in the lesson (i.e., oriented toward teacher). 
The total number of children engaged was then recorded. Following lesson implementation, the 
total number of engaged children during each one-minute interval was divided by the number of 
children present. The percentage of children engaged across all of the one-minute intervals was 
averaged to create an overall child engagement variable across all observations. 
Analyses 

All analyses were conducted in Stata version 14. Multivariate regression analyses that 
used the generalized Huber-White sandwich estimator to adjust standard errors for non- 
independence (clustering by classroom) were conducted to test the effects of PA on children’s 
social-emotional competence and health behaviors. In each analysis, standardized post-test 
scores were regressed on condition (0 = control, 1 = intervention), pre-test scores, child age, 
gender, and race (race coded as white versus non-white). Post-test scores were standardized in 
order to interpret the estimated standard deviation change in the outcome (i.e., a standardized 
effect size). Additionally, full information maximum likelihood estimators were used to take 
advantage of all available data, which provide less biased estimates that listwise deletion. 

Although no significant differences emerged between groups at baseline with regard to 
children’s pre- or post-intervention outcomes or control demographics, there were significant 


baseline differences between children in the control group and children in the intervention group 
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on two variables: half-day versus full-day classrooms and Head Start versus non-Head Start 
programs. However, we did not include these variables as baseline covariates in our models 
because these they were highly confounded with the intervention variable. Specifically, the large 
majority of children in the intervention attended Head Start (89.47%), which were all half-day 
programs. The small sample size and lack of coverage of program type across groups prevents 
accurately estimating the contributions of these two different effects (i.e., preschool type and 
classroom time) independent of the intervention assignment. Therefore, effect sizes (ES) 
presented are the predicted intervention effect on the standardized post-test score conditional on 
children’s pre-test score, sex, age, and race (all of which did not significantly differ by 
condition). 

Descriptive statistics were used to analyze fidelity and engagement data. We only report 
on descriptive statistics rather than inferential regression estimates due to a lack of power as the 
fidelity sample was restricted to only children in the PA program (” = 15 — 33). The regression- 
estimated effects of student engagement, teacher implementation, and teacher attitudes on 
outcomes for children in PA are available by request. 

Sensitivity and attrition analyses. As a sensitivity analysis, we present regression 
models in which standardized post-test scores were regressed on condition and pre-test scores 
only and there were no adjustments for clustering or demographics. We also present independent 
samples t-tests on child post-test scores (unstandardized) as the unconditional program effects. 
We tested whether any outcomes or demographics differed by condition. There were few 
significant demographic differences (white children had more post-intervention reports 
completed) and, for outcomes, there were significantly fewer teacher reports at post-intervention 


for the control group, which may be due to one control teacher not completing any post- 
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intervention reports. 

Post-hoc power analysis. We focus primarily on the substantive significance rather than 
the statistical significance in part because the study was underpowered. When calculating power, 
we considered how variations in the sample size and effect size would influence the likelihood of 
finding statistically significant results. For example, if the true effect size was .25 and our sample 
was 75 (the number recruited for participation), we would only detect a statistically significant 
effect 19% of the time. Furthermore, if the true effect size was actually .50 and the sample was 
75, we would still only detect a statistically significant effect 56% of the time. Finally, due to 
missing data on parent reports, some outcomes had as few as 27 observations. In this scenario, 
our likelihood of reaching statistically significance drops to 9% of the time for a true effect size 
of .25 and 23% of the time for a true effect size of .50. Thus, statistical significance and 
substantive significance are both considered when interpreting our results. 

Results 
Descriptive analyses 

Correlations between measures at time | (pre-test) and time 2 (post-test) are reported in 
Table 1. In general, correlations among measures were higher among common assessors (e.g., 
teachers) than among common constructs (e.g., Student Rating Scale). The direct assessment of 
Aggressive Problem Solving was not significantly correlated with parent or teacher ratings, with 
the exception of parent ratings of the Student Rating Scale at time 1 (7 = -.24, p < .001). The pre- 
and post-test means and standard deviations for the control and the PA groups are shown in 
Table 2, as well as t-tests for differences at each time point. Children in the PA and control group 
did not significantly differ on any of the baseline measures of social-emotional competence and 


health behaviors. At post-test, children in the PA group were marginally higher on parent-rated 
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health behaviors, but also marginally lower on teacher-rated social skills. 
Effects of PA on Social-Emotional Competence and Health Behaviors 

Results demonstrated program effects in the intended directions for the two parent-rated 
measures and the direct assessment (see Figure 1), although none of these findings were 
significant at the a = .05 level. That is, children who participated in the PA program showed 
greater improvements on the parent-rated Student Rating Scale compared to children in the 
control group (effect size [ES] = .70, standard error [SE] = .42). Parents rated children who 
participated in PA as roughly two-thirds of a standard deviation better on the Student Rating 
Scale (which included health behaviors) at post-test than children in the control group. 
Furthermore, children who participated in PA showed greater improvements on the parent-rated 
Health Behaviors Scale compared to the control group (ES = .29, SE = .24). That is, parents rated 
children in the PA condition as engaging in more healthy behaviors at post-test compared to 
children who were not in PA, roughly equivalent to one-third of a standard deviation increase in 
healthy behaviors. Finally, children who participated in PA had fewer aggressive responses on 
the Social Problem Solving direct assessment compared to the control group (ES = -.29, SE = 
.30). This effect is equivalent to roughly one quarter of a standard deviation fewer aggressive 
responses on situational vignettes. 

Unexpectedly, and inconsistent with the parent ratings and direct assessment, the teacher 
ratings showed iatrogenic (i.e., non-beneficial) program effects (see Figure 1). Children who 
participated in PA showed fewer improvements on the teacher-rated Student Rating Scale (ES = - 
.24, SE = .25), equivalent to roughly one quarter of a standard deviation worse. Children who 
participated in PA showed fewer improvements on the teacher-rated SSIS-RS (ES = -.54, SE = 


.42), equivalent to roughly half of a standard deviation worse. Finally, children showed increases 
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in behavior problems as assessed by the SSIS-RS (ES = .12, SE = .36). Therefore, from the 
teachers’ perspective, children who participated in the PA program generally demonstrated 
poorer social-emotional outcomes than control children. However, teachers were aware of 
program assignment whereas direct assessors and parents were not. 

Sensitivity analyses. The sensitivity analyses revealed similar results. However, two of 
the six models were statistically significant at the a = .05 level: teacher-rated SSIS-RS (ES = - 
51, SE = .24, p = .04) and parent-rated Student Rating Scale (ES = .65, SE = .29, p = .03). 
Independent samples t-tests revealed no significant results. 

Fidelity of Implementation and Child Engagement 

On average, over the course of 15 weeks, teachers reported delivering nearly four lessons 
each week (M = 3.50, SD = .41) that ranged from 10-15 minutes each, which was the targeted 
duration and dosage. Teachers also indicated that, on average, they implemented the lessons as 
intended (M = 3.22, SD = .38; between often and always on the scale). Furthermore, teachers 
largely reported very positive attitudes regarding the PA program and its utility (M = 3.63, SD = 
.18; between agree and strongly agree). Observations of implementation fidelity generally 
mirrored those of teacher reports. Across seven observations, observers rated children as being 
highly engaged in PA lessons the majority of the time (mean percentage of child engagement = 
84.31%, SD = 3.99; range = 80.5% - 91.49%). Moreover, on average, adherence and quality of 
implementation scores from observations were high (M = 2.40, SD = .28 on a 3-point scale). In 
general, teachers reported implementing the program with fidelity, teachers reported positive 
attitudes toward the program, and children were engaged in the PA lessons. 

Discussion 


In the present study, mixed results emerged regarding the effectiveness of PA on social- 
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emotional outcomes and health behaviors for children primarily from low-income families. 
Although not statistically significant, parents reported improvements on the Student Rating Scale 
and health behaviors for children who participated in PA compared to a control group. Further, 
results indicated that the PA program decreased aggressive responses to the social problem 
solving direct assessment (though not statistically significant). In contrast, the PA program 
effects on teacher ratings were in the iatrogenic direction, though also not statistically significant. 
Teachers rated children who participated in PA as having poorer social-emotional skills 
compared to a control group. 

Both observed and teacher-rated fidelity results indicated that PA teachers implemented 
the intended number of lessons, generally taught lessons as intended, and had positive attitudes 
about the program. Furthermore, observed child engagement was high during lesson 
implementation. This study provides evidence that the PA preschool program is feasible for early 
childhood teachers and may be effective in promoting positive social-emotional and health 
behaviors; however, this study also calls attention to potential issues with measurement and 
study design when evaluating classroom-based interventions. 

PA Effects on Social-Emotional Competence and Health Behaviors 

Results from analyses using parent reports and a direct assessment suggest that the PA 
program shows preliminary evidence for improving children’s social-emotional competence. 
Although not statistically significant, the effect sizes were comparable to those documented in 
other intervention studies focused on social-emotional development (Bierman et al., 2008; 
Durlak et al., 2011), and the effect size for the Student Rating Scale exceeded those reported in 
previous studies. These findings are important given empirical evidence for pervasive gaps in 


these domains for children from high-risk populations (Qi & Kaiser, 2003). Indeed, research 
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suggests that children from low-income families often enter kindergarten with lower levels of 
social-emotional functioning (Qi & Kaiser, 2003), and poorly developed social skills in early 
childhood can have long-term implications for children’s development (Denham et al., 2003). 
Identifying effective interventions that can be implemented prior to formal school entry may be 
critical for alleviating disparities and potentially improving developmental trajectories for 
children from disadvantaged backgrounds. 

In contrast to parent reports and direct assessment, results showed negative effects 
(though not statistically significant) on teacher-reported social-emotional competence as a result 
of PA participation. These findings are inconsistent with the previous evaluation of the PA 
preschool program that demonstrated positive effects of PA lessons on children’s teacher-rated 
social-emotional skills (Schmitt et al., 2014). There are a few possible explanations for these 
discrepant results. For instance, in the first evaluation (Schmitt et al., 2014), teachers were not 
randomly assigned to implement the intervention as they were in the current study. This self- 
selection could have resulted in teachers being more favorable to the program, and perhaps, over- 
reporting positive effects. Indeed, unlike the current study, there were significant pre-test 
differences on teacher ratings of children’s social-emotional skills in the Schmitt et al. (2014) 
study, suggesting children were not equivalent across groups at baseline. 

Another potential explanation is with regard to teacher training prior to PA 
implementation. In the first evaluation, teachers did not receive training from PA staff on the 
program or on how to implement it. In the current study, teachers assigned to implement the 
intervention attended a 3-hour training workshop on the PA program delivered by PA staff. This 
training focused on how to effectively implement the PA lessons, as well as previous research 


and evaluations of PA and associated outcomes. Including previous research as part of the 
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training was intended to enhance teacher buy-in of the program, although it is possible that it 
created a bias in teachers that had an inadvertent effect on their ratings of children. Specifically, 
reference bias may have emerged, such that following training, intervention teachers may have 
upwardly redefined their definition of what strong social-emotional skills look like in early 
childhood and what sort of impact PA would have on these skills and thus, rated children more 
critically. Indeed, this type of reference bias has emerged in other literature looking at self-report 
(West et al., 2015). Similarly, teachers may have been more sensitive to, and aware of, the 
behaviors the program intends to improve as a result of explicitly teaching the skills. This 
sensitivity and awareness may have resulted in teachers noticing the behaviors/skills (positive 
and negative) in the classroom more than teachers who were assigned to the control condition 
and, in turn, reporting them more or less frequently on the surveys. 

Finally, although control teachers were given access to intervention materials following 
the study, they may have unintentionally rated children as being higher or lower on certain skills 
as a result of compensatory rivalry (Shadish, Cook, & Campbell, 2002), or believing that despite 
not getting an intervention, children in their classrooms were showing improvement in social- 
emotional skills. As in any evaluation, caution should be exercised when utilizing ratings of 
children’s behavior that are completed by the same people who delivered the intervention. 
Evidence suggests stakeholders should be particularly cautious of using teacher report in 
evaluation work as these reports come with a host of limitations, including misinterpretation of 
items, lack of accuracy, and social desirability bias (Duckworth & Yeager, 2015). 

Notably, the PA program is one of the first early childhood interventions designed to 
improve social-emotional competence and health behaviors simultaneously, and results suggest 


that participation in PA has the potential to improve children’s health behaviors over the course 


20 


POSITIVE ACTION, SOCIAL-EMOTIONAL SKILLS, AND HEALTH BEHAVIORS 


of the preschool year. Physical health is an important component of school readiness, and 
although early childhood programs often include services related to certain aspects of child 
health (e.g., hearing screenings), promoting healthy behaviors is often overlooked in early 
childhood intervention efforts. Explicitly teaching children positive health behaviors in preschool 
may be important as this is when many behavioral health patterns are being established. 
Fidelity of Implementation 

Descriptive results from teacher ratings and independent observations indicated that 
teachers implemented the program with fidelity and had positive attitudes toward the program, 
and that children were engaged in the activities. These findings are important because the quality 
with which teachers are implementing the lessons and the extent that children are engaged with 
PA content can have an impact on whether the intervention will positively affect the intended 
outcomes (Pettigrew et al., 2015). Indeed, previous research suggests that when teachers can 
feasibly implement a classroom-based intervention focused on social-emotional functioning, 
children are more likely to demonstrate improvements in these skills (Durlak et al., 2011). 
Limitations 

Although this study has a number of strengths relative to the previous preschool PA 
evaluation, limitations must be noted. First, our sample was relatively small, particularly for 
post-program parent reports. The small sample size limited the analyses that could be conducted 
and created large standard error estimates. Furthermore, differences between the intervention and 
control groups were present on program type (i.e., Head Start versus non Head Start; full-day 
versus half-day) indicating that the randomization may have failed to distribute differences 
equally across conditions. Importantly though, children did not significantly differ by condition 


on pre-test measures of social-emotional competence and health behaviors or demographic 
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characteristics. It will be critical for future PA evaluations to recruit larger samples to confirm 
that results hold across different types of early childhood programs. 

Another limitation worth noting is that we found low correlations between teacher and 
parent reports of the same measure. That is, construct validity is called into question when 
measures administered by different assessors but intended to tap the same, or similar, constructs 
are unrelated. For example, parent and teacher ratings of the Student Rating Scale were only 
correlated at .36 at pretest and .07 at posttest. Furthermore, the direct assessment of aggressive 
behaviors was only significantly related to parent-ratings on the Student Rating Scale at pretest, 
and unrelated to all other social-emotional competence measures. Finally, similar to potential 
issues of relying on teacher reports noted above, there may also be limitations when using parent 
reports as well, including small reference groups and reporter bias. Measurement issues of 
social-emotional skills are a pressing concern for the field of early childhood development and a 
need for well-validated, reliable measures is necessary to better understand intervention effects 
and to generalize findings across studies (Jones, Zaslow, Darling-Churchill, & Hallie, 2016). 
Conclusion 

This study provides preliminary evidence that the PA preschool lessons may improve 
children’s parent-rated social-emotional competence and health behaviors, as well their social 
problem solving skills. Further, results suggest that teachers can implement PA with fidelity and 
that children enjoy the lessons. Children from low-income families may be at elevated risk for 
poor social-emotional functioning and health (Qi & Kaiser, 2003), and thus, identifying feasible 
and effective interventions aimed to improve these important components of school readiness is 
critical. The PA preschool program deserves continued refinement, and more evaluation research 


is needed to unpack these promising, yet conflicting results. 
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Ethical approval: All procedures performed in studies involving human participants were in 

accordance with the ethical standards of the institutional and/or national research committee and 
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